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© VISUAL CONTROL ROBOT SYSTEM. 



y^© This invention relates to a visual control robot 
Ifj system for arc welding which can conduct easily and 
^•efficiently those operations which are associated with 
l^a robot controller and a vision sensor system by an 
|V operator through a common operation panel. Before 
OOthe start of welding, a soft key (9a) disposed on the 
q common operation panel (6) for both the robot con- 
troller (5) and vision sensor controller (3) is operated 



UJ 



so as to set the robot system to a first mode and 
under this state, the position of a sanTple welding 
member is taught to the robot controller. The soft 



key Is then operated to switch an operation mode to 
a second mode in order to input parameters for 
vision teaching of storing a camera image at each 
point on a reference weld line formed on the sample 
weld member. The position teaching and the vision 
teaching can be given via the same operation panel 
and moreover, the operation common to both can be 
carried out by the same operation. Correction control 
is effected during the welding work so that an actual 
weld line is in agreement with the reference weld 
line in accordance with the result of comparison 
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between an actual camera image representing a 
welding state and a camera image at the time of 
vision teaching corresponding to the former. 
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SPECIFICATION 
VISUALLY CONTROLLED ROBOT SYSTEM 
Technical Field 
The present invention relates to a visually 



5 controlled robot system in which a robot control device 
and a vision sensor system can be individually operated 
through a common operator panel. 

Background Art 
There have been known visually controlled robot 

10 systems operated in accordance with visual information 
detected by a vision sensor. As a typical example, in 
visually controlled robots for arc wadding, slit light 
is radiated from a laser beam projector of the vision 
sensor to a member to be welded, and a slit image is 

15 detected by a semiconductor camera of the vision 

sensor. During a welding process, a vision sensor 
controller of the robot determines the position of an 
actual weld line by analyzing the slit image and 
detects a deviation of the actual weld line from a 

20 reference weld line formed on ^ sample weld member for 
vision teaching, and the robot control device performs 
a correction in accordance with the detected deviation 
so that the actual weld line and the reference weld 
line coincide, while carrying out an arc welding. 

25 Here, the vision teaching denotes a process of storing 
camera pictures obtained with a laser beam sequentially 
radiated to a plurality of points on the reference weld 
line formed beforehand on the sample weld member when a 
welding operation is stopped, in the vision sensor 

30 controller in relation to robot operated positions. 

The slit images in the camera pictures obtained at the 
individual points on the weld line of the sample weld 
member exhibit characteristic portions representing the 
position of the weld line. For example, in the case of 
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a lap joint, the slit image contains a disconnected 
portion as a characteristic portion, and in the case of 
a fillet welded Joint, the slit image contains a bent 
point as a characteristic portion. Thus, the 
characteristic portions of the slit images in a 
plurality of camera pictures cooperatively indicate the 
reference weld line. 

When carrying out a vision teaching, the operator 
must input various parameters for the vision teaching 
to the vision sensor controller. Therefore, 
conventionally, an operator panel used exclusively for 
the robot control device to teach actions to the robot 
and an operator panel used exclusively for the arc 
vision sensor system are separately provided. 
Accordingly, the operator must be skilled in the 
operation of these two operator panels and must operate 
the panels alternately, suffering a laborious and 
inefficient operation. 

Disclosure of the Invention 
An object of the present invention is to provide a 
visually controlled robot system which permits an 
operator to easily and efficiently carry out operations 
individually related to a robot control device and a 
vision sensor system through a common operator panel. 

To achieve the above object, according to the 
invention, there is provided a visually controlled 
robot system comprising a robot control device for 
controlling an operation of a robot body, a vision 
sensor system for detecting visual information, and a 
common operator panel having means for selecting one of 
a first mode for carrying out an operation related to 
the robot control device and a second mode for carrying 
out an operation related to the vision sensor system, 
to carry out the operations. 
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According to the present invention, operations 
individually related to the robot control system and 
vision sensor system of the visually controlled robot 
system can be carried out through the common operator 
5 panel, and therefore, the operator can easily perform 
an operation related to the robot control device and an 
operation related to the vision sensor system by simply 
switching the operation modes of the robot system, 
using the only one operator panel, whereby the 

10 operating efficiency is improved. Furthermore, the 

operator can easily get skilled in the manipulation of 
the operator panel. 

Brief Description of the Drawing 
The single figure is a diagram showing, partly in 

15 blocks, a visually controlled robot system for arc 
welding according to an embodiment of the invention. 
Best Mode of Carrying Out the Invention 
In the figure, a visually controlled robot system 
for arc welding comprises a robot body 1, and a welding 

20 torch 2a and an arc vision sensor 2b, which are mounted 
to a wrist portion 2 of the robot 1. The vision sensor 
2b constitutes an arc vision sensor system in 
cooperation with an arc vision sensor controller 3 
having a CRT display 4, and is provided with a las^r 

25 beam projector including a laser oscillator, and a 
semiconductor camera (none of them are shown) * The 
welding torch 2a and the projector and camera of the 
sensor 2b are always in a predetermined positional 
relationship relative to one another. The robot body 1 

30 is provided with servomotors (not shown) serving as 
means for driving individual axes of the robot, and 
various internal sensors (not shown) for detecting the 
operated positions of the servomotors for the 
individual axes. The servomotors and the internal 
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sensors are connected to a robot control device 5 
comprising, e.g., a numerical control device. 

The robot system is further provided with an 
operator panel 6, which is connected to the robot 
control device 5 and also connected through the control 
device 5 to the arc vision sensor controller 3 such 
that various operations individually related to the 
control device 5 and the vision sensor system can be 
carried out through the common operator panel 6, 
Namely, the robot system is operable in various modes 
including a first mode for carrying out au operation 
related to the robot control device 5 and a second mode 
for carrying out an operation related to the arc vision 
sensor system (more specifically, the arc vision sensor 
controller 3), and the operator panel 6 is provided 
with means for selecting one of these modes. 
Specifically, the operator panel 6 comprises a liquid- 
crystal display section 7, various keys/buttons 8 
including cursor keys and ten-key pad, and a plurality 
of software keys 9. The software keys 9 have different 
functions in different modes of the robot system, 
whereby a large number of functions can be executed. 
For example , a predetermined software key 9a is 
designed to function as the aforementioned mode 
selection means when the robot system is powered. 
Moreover, the robot system is constructed such that 
operations common to both the first and second modes 
(e.g., an operation for moving the cursor on the 
screens of the CRT display 4 and the liquid-crystal 
display section 7, etc.) can be executed by operating 
the keys/buttons of the operator panel 6 in the same 
manner . 

Now, the operation of the arc welding robot system 
constructed as above will be described. 



EP 0 377 755 A1 



- 5 - 

In the robot system, teaching of the position of a 
weld member, teaching of welding conditions, vision 
teaching, and an actual welding process are executed in 
the order mentioned. 
5 First, the operator sets a sample weld member on a 

jig (neither is shown) and turns on the power of the 
robot system. Upon application of the power, the arc 
vision sensor controller 3 causes the liquid-crystal 
display section 7 of the operator panel 6 to display a 

10 mode selection screen. In this screen, various 

functions imparted to the respective software keys 9 at 
the time of the mode selection are displayed. The 
operator operates the predetermined software key 9a 
which then functions as the mode selection means, to 

15 select the first mode for carrying out an operation 
related to the robot control device 5. 

Next, by pressing the various keys/buttons 8 of 
the operator panel 6, the operator manually operates 
the robot body 1 through the robot control device 5 to 

20 teach the position of the sample weld member 
(specifically, the operated positions of the 
servomotors for the individual axes detected by the 
internal sensors) to the robot control device 5. Then, 
the operator carries out a tentative welding on the 

25 sample weld member, and sets the resultant proper 

welding conditions obtained in the robot control device 
5. 

Subsequently, to carry out the vision teaching, 
the operator presses the software key 9a of the 
30 operator panel 6 to display the mode selection screen 
at the liquid-crystal display section 7, and selects 
the second mode for carrying out an operation related 
to the arc vision sensor system. In accordance with 
this mode selection, the arc vision sensor controller 3 
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sequentially displays messages questioning about the 
vision teaching, at the liquid-crystal display section 
7. The operator presses the keys/buttons 8 of the 
operator panel 6 in answer to the questions, to thereby 
5 set parameters relating to the vision teaching, such as 
a region for image analysis on the screen of the CRT 
display 4, a type of a joint, an image acquisition 
interval, etc. r . 

In accordance with the vision teaching parameters 

10 thus set, the vision sensor controller 3 cooperates 

with the robot control device 5 to carry ^>ut a vision 
teaching operation. At the time of the vision 
teaching, a welding process is stopped. When the 
vision teaching is carried out, the robot control 

15 device 5 controls the operation of the robot body 1 in 
accordance with the teaching on the position of the 
aforesaid sample weld member, such that the welding 
torch 2a attached to the wrist portion 2 of the robot 
body 1 is moved along the reference weld line formed on 

20 the sample weld member. During this time, slit light 
with a linear section extending in the direction of 
traversing the weld line of the sample weld member is 
radiated from the laser beam projector of the vision 
sensor 2b toward the sample weld member, and a slit 

25 image (a V-shaped bent line, if the sample weld member 
is a fillet welded joint) is detected by the 
semiconductor camera of the sensor 2b. The slit image 
is displayed on the screen of the CRT display 4 
associated with the vision sensor controller 3 

30 connected to the semiconductor camera. The vision 

sensor controller 3 reads camera pictures including the 
slit images into a storage device incorporated therein 
at predetermined intervals of time (corresponding to a 
distance of 2 to 50 mm on the weld line) and in 
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relation to the position and orientation of the wrist 
portion 2 of the robot body 1. 

As mentioned above, during individual operations 
in the first snd second modes, the operator carries out 
5 a necessary operation by using only the operator panel 
6 common to these modes, and operations common to these 
two modes are effected by operating: the keys/buttons of 
the operator panel 6 in the same manner, whereby even 
an unskilled person can easily and efficiently carry 

10 out an operation in these two modes. 

After the above-described various operations are 
finished, the operator sets a weld metaber which is to 
be actually welded, and operates the operator panel 6 
to put the robot system in a welding operation mode, 

15 thereby starting a welding process. 

During an arc welding process, the robot control 
device 5 controls the position and orientation of the 
wrist portion 2 of the robot body 1 (i.e., the position 
and orientation of the welding torch 2a) in accordance 

20 with the teaching on the position of the aforesaid 
sample weld member, read from the built-in storage 
device, and carries out a welding under the proper 
welding conditions determined in the aforesaid manner. 
In this welding process, slit light is radiated from 

25 the laser beam projector of the vision sensor 2b toward 
the weld member, as in the vision teaching process, 
whereby a current welded state is imaged by the camera 
of the sensor 2b and displayed on the CRT screen. The 
vision sensor controller 3 reads the camera picture 

30 when the welding torch 2a reaches each of predetermined 
points on the weld line, and also reads out a camera 
picture whose torch position corresponds to that of the 
currently read picture, from among the camera pictures 
stored in the built-in storage device at the time of 
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the vision teaching. Next, the vision sensor 
controller 3 carries out an image analysis on each of 
the two camera pictures and extracts characteristic 
portions of the slit images contained the camera 
5 pictures which represent the weld line (e.g., a bent 

point of the V-shaped bent line if the weld member is a 
fillet welded joint). Then, the controller 3 
calculates the magnitude of the on-screen deviation 
between the characteristic portions of the two camera 

10 pictures, and calculates the actual dimensions 

(correction amounts) of the deviation between the 
reference weld line used at the vision teaching and the 
actual weld line for the welding process, based on the 
calculated magnitude value and a scale factor 

15 determined in accordance with the known distances 

between the laser beam projector and the camera and 
between the projector and the welding member. The 
deviation of position is caused by a setting error of a 
workpiece, shifting of the position of the workpiece 

20 during a welding process, etc. The vision sensor 
controller 3 then sends the aforesaid correction 
amounts to the robot control device 5, and in 
accordance with the correction amounts, the robot 
control device 5 carries out a real-time correction of 

25 the weld path. This correction is effected at each of 
the points on the weld line, whereby arc welding can be 
carried out with accuracy. 
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CLAIMS 

1. A visually controlled robot system comprising: 
a robot control device for controlling an 

operation of a robot body; 
5 a vision sensor system for detecting: visual 

information; and 

a common operator panel having means for selecting 
one of a first mode for carrying out an operation 
related to said robot control device and a second mode 
10 for carrying out an operation related to said vision 
sensor system, to carry out said operations. 

2. A visually controlled robot *sy stem according 
to claim 1, wherein said vision sensor system includes 
a vision sensor for imaging at least a subject of work, 

15 and a vision sensor controller for analyzing a picture 
signal from said vision sensor, said vision sensor 
controller being connected to said operator panel. 

3. A visually controlled robot system according 
to claim 1, wherein an operation common to said first 

20 and second modes is executed through an identical 
operation of the operator panel by an operator. 

4. A visually controlled robot system according 
to any one of claims 1 to 4, wherein a welding torch is 
mounted to said robot body to carry out arc welding. 
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